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An empirical study of personal response technology for improving 
attendance and learning in a large class 
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Abstract: Student evaluations of a large General Psychology course indicate that 
students enjoy the class a great deal, yet attendance is low. An experiment was 
conducted to evaluate a personal response system as a solution. Attendance rose 
by 30% as compared to extra credit as an inducement, but was equivalent to 
offering pop quizzes. Performance on test items targeted by in-class questions 
rose by an average of 21% while control test questions rose by only 3%. The 
effect is seen in both factual and conceptual test items. Two theories that may 
explain the effect are discussed. 
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I. Introduction. 

It’s a problem familiar to many professors. Each semester my students wrote glowing 
reviews of my General Psychology course, rating the class a mean of 4.5 on a 5-point scale. In 
spite of the good reviews, large numbers were absent from class on any given day. This 
mismatch between students’ perceptions and behaviors is most often found in large classes 
where attendance is not easily monitored and students are largely anonymous. Indeed, the 
problem became pronounced after my class size increased from 60 to 210 students. The student 
evaluations remained very positive but attendance was reduced to roughly 50-60% on any given 
day. Moreover, when students were in attendance many were inattentive, either dozing or 
otherwise occupied for at least part of the class period. In speaking with colleagues at campuses 
across the country, I found the problem to be common. The attendance and attention problem is 
directly related to learning because students aren’t learning the classroom material if they aren’t 
in class. I increased attendance to roughly 80% by giving pop quizzes throughout the semester. 
The system was cumbersome, however, as handing out and collecting tests from hundreds of 
students took a lot of time away from class. Grading them and inputting scores to grade books 
also proved time consuming. 

I found a partial solution, however, in the use of a personal response system (PRS). PRS 
facilitates presentation of multiple-choice questions into any class equipped with a digital 
projection system. Because I was already using PowerPoint to present all my lecture material, the 
technology integrated naturally into my classroom. PRS requires students to purchase a remote 
(commonly called a “clicker”) that allows them to “click in” responses, which are recorded by a 
receiver. Questions can be used to check comprehension or promote discussion. With the 
instructor’s remote, a button click allows instant projection of class responses to provide 
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immediate feedback. Uploading students’ responses to a grade book also requires just a simple 
button click by the instructor. 

PRS use in large classes has become very common nationwide but research on its 
learning effects is both sparse and inconclusive. Some investigations show positive effects of 
PRS on learning while others do not. As I will show in the following section, the underlying 
reason for differential findings may stem from methodological issues. For this reason, the present 
investigation focuses on the attendance and learning effects of PRS. It uses a methodology that 
offers a more fine-grained view of the learning effects in the hope of explaining and clarifying 
some of the literature’s contradictory results. 

A. Research on PRS Ejfectiveness. 

PRS systems have been used for a variety of purposes including teaching case studies 
(Herried, 2006; Brickman, 2006), replicating published studies in class (Cleary, 2008) and 
electronic testing (Epstein, Lazarus, Calvano, Matthews, Hendel, Epstein and Brosvic, 2002). 
Based on published reports, however, the most common use appears to be during lectures for 
assessing students’ comprehension of class material in real time and improving participation and 
attendance (Beekes, 2006; Poirier and Eeldman, 2007; Shih, Rogers, Hart, Phillis and Lavoi, 
2008). The latter function has also been the focus of more scrutiny. 

A number of studies have attempted to test the effect of PRS on attendance and 
participation. Student volunteers using PRS in a controlled laboratory study were significantly 
more likely to participate than students asked to raise their hand or use laminated response cards 
to indicate responses to instructor questions (Stow ell and Nelson, 2007). In a case study of PRS 
in a large introductory biology course, Ribbens (2007) reported an attendance increase of 20% 
after the technology was introduced in his course. PRS was also shown to enhance student 
participation in classes as part of an institution-wide evaluation across disciplines (Draper and 
Brown, 2004). One of the strongest effects of the technology in that study was its ability to 
promote class discussion among students. The increased participation may come, in part, from 
student enjoyment of the technology. Indeed, a common finding among PRS studies is that 
students enjoy the technology in class. Eor example. Hatch, Jensen and Moore (2005) report that 
96% of students enrolled in their anatomy and environmental science courses agreed or strongly 
agreed that they liked using the technology. It is highly likely, though, that participation effects 
also stem from using PRS to determine required participation grades. 

Not all reports have shown a clear improvement in student participation, however. 
Morling, McAuliffe, Cohen and DiLorenzo (2008) compared outcomes of 2 classes using PRS 
with 2 classes that were not. Two instructors each taught one PRS class and one no-PRS class. In 
the PRS classes, the technology was used at the start of each class to quiz students on assigned 
readings. They found that one instructor’s PRS class rated attendance as more important than the 
no-PRS class, but ratings were comparable between the other instructor’s classes. Neither PRS 
class reported being more engaged or attentive in class than their matched no-PRS class. It is 
important to note, however, that the PRS questions were scored for extra credit and not as a 
required component of the course. Moreover, PRS use in this study was limited to tests given at 
the start of class and only probed memory for the assigned reading, not for in-class material. 
There is no reason to expect that testing students about outside reading in the beginning of class 
would cause students to be more attentive or interested in lecture material during class. Indeed, 
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in a discussion of recommended best practices, Ribbens (2007) suggests integrating PRS 
throughout the class and using it as part of the graded requirements. 

The learning benefit of PRS is another important issue that has been addressed in the 
empirical literature. In an assessment of students’ self-perceptions of learning, Hatch et al. 
(2005) report that 92% of students indicated PRS helped them identify what they did and did not 
know and 83% indicated that the technology helped them learn. Of course, student’s self- 
perceptions are not as accurate as more direct measures. In one study more directly measuring 
the effect, Ribbens (2007) found that students in his introductory biology course did 8% better on 
tests than his class 2 years prior, before adopting PRS. Morling et al. (2008) also reported higher 
mean test scores on 2 of 4 tests in their PRS classes than in their no-PRS classes. Morling et al.’s 
study is nicely controlled by its use of 4 classes counterbalanced between instructors and control 
groups. However, the authors do not mention how many of the test questions were directly 
related to the information targeted by the in-class PRS questions. The same question arises about 
Ribbens’ (2007) findings. That information would be useful to understanding the extent of PRS 
effect on students’ learning and understanding, as performance on questions not targeted by PRS 
may be diluting the dependent measure. 

Other investigations have yielded somewhat mixed results in their analyses of learning 
with PRS. For example, Kennedy and Cutts (2007) found that the strength of the relationship 
between PRS use and learning outcome measures hinged on how successful students were in 
answering the PRS questions. Others have found no learning effects of PRS at all. Stowell and 
Nelson (2007) gave laboratory subjects a simulated introductory psychology lecture and 
compared test performance between groups asked to use PRS or do other sorts of participative 
activities during the lecture. They found no differences between groups on learning outcomes 
measures. Of course, the study was conducted in a laboratory so motivation to learn was 
different than in a live classroom. 

In sum, the majority of studies on PRS point to the technology as an effective 
pedagogical tool and methodological issues appear to be a factor in those that do not. 
Specifically, assessments of PRS do not always assure internal validity through careful control of 
the relationship between in-class PRS use and the dependent measure. The hypothesis explored 
in the present study is that learning measures targeted at specific PRS questions will demonstrate 
a strong effect of the technology on learning. Before describing the study, the following section 
explains more about PRS and how it was incorporated into my classroom. 

B. PRS Use in the Present Classroom. 

My General Psychology class is typical of most, covering roughly one chapter of a 
textbook each week and spanning a cross section of the field. I use demonstrations, role-playing, 
audio, video, and interactive activities to demonstrate points. All of my lectures are presented 
with PowerPoint slides that are available on my website for students to download. 

The system used in the present study was iClicker. The devices cost students $20-35 
(depending on whether it came bundled with a text or was bought used). The clickers were 
available at the campus bookstore or through Amazon.com. A receiver connected to the 
instructor’s computer registers the responses. The iClicker company provides the instructor’s 
hardware (a receiver and 2 instructor remotes) and software at no cost. The software runs 
concurrently with PowerPoint, with a small function box floating on top of the slides in a place 
of the user’s choosing. It allows a bar graph of the class responses to be displayed 
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instantaneously. Record keeping is also automated, as students’ scores are uploaded to a grade 
book within seconds or entered into a text only file that is transferred easily to an Excel file or 
Blackboard grade book. Questions can be ungraded, assigned participation points for entering 
any response or assigned points for correct responses only. Earned points can be factored into 
final grade calculations or used for extra credit. 

In my class, I present roughly 50-70 credited, multiple choice questions over the course 
of the semester. The accumulated PRS points are scaled to a maximum score of 50 and 
calculated into the final grade as 50 out of a possible 350 points. Other questions, however, are 
not scored and are used solely to make a point or generate discussion. I typically present graded 
PRS questions after making an important point, explaining a theory or presenting a research 
finding, but only after soliciting questions and encouraging students to ask for clarifications. 
Some questions are factual (e.g.. What is the major dijference between a punishment and a 
negative reinforcer?) while others are more conceptual, requiring students to apply a principle 
(e.g.. Given what we know about the role of proximity and similarity in our attraction to others, 
in which setting are you least likely to meet a new friend or your future spouse?). The PRS 
question slides are omitted from the download files I make available to students. 

The purpose of incorporating PRS into the course this way was to improve attendance 
and to enhance student learning. The study presented here was conducted to evaluate the 
effectiveness of the approach. While the research summarized earlier was encouraging enough to 
try out PRS in my class, it also convinced me that evaluations of PRS are particularly sensitive to 
variables affecting external and internal validity. To maximize external validity, the study was 
conducted in a live classroom. To maximize internal validity, I focused on the relationship 
between the PRS questions and the assessment items during stimulus development. I also used 
control items and control groups from prior semesters. Specifically, the effect on learning was 
measured by pairing in-class PRS questions with specific test questions. Performance on the 
targeted test questions was compared with test questions that were not paired with PRS items. In 
addition, performance on the same test items in a prior semester that did not include PRS was 
used as a baseline measure. The methodology is explained in detail below. 

II. Method. 

A. Subjects. 

Students enrolled in a 210-student General Psychology (PSYlOl) class at the University 
of Massachusetts Dartmouth during fall 2007 comprised the experimental group. All but a 
handful was traditional students, aged between 18-21. The majority, 81%, were freshmen, 14% 
sophomores, 4% juniors and 1% seniors. Because the course satisfies a university-wide 
distribution requirement as well as requirements within several majors, students came from all 
five campus colleges. 29% of the class was business majors, 23% nursing, and 40% liberal arts 
and sciences. The rest were disctibuted between engineering and visual and performing arts. 
Attendance and test scores of students registered in fall 2006, prior to the implementation of 
PRS, were used as baseline measures to evaluate the performance of the fall 2007 class. The 
majority were freshmen and sophomores, 84% and 14%, respectively. They represented all 5 
colleges, with the bulk coming from business, nursing and social sciences/humanities, 43%, 
16%, and 36%, respectively. One other attendance measure was used from a class in fall 2005. 
Students in that class were similar to the others in distribution of academic years (68% freshmen. 
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22% sophomores and the rest juniors and seniors) with most majoring in business, nursing and 
liberal arts and sciences, 21%, 3% and 58%, respectively. An IRB waiver was obtained prior to 
conducting the study. 

B. Stimuli, Materials and Procedure. 

The course taught to the PRS class was almost identical to the course taught to the No- 
PRS class, including the assigned text, all lectures and PowerPoint slides, projected with an 
Apple iBook G4. The difference was the addition of the PRS questions in the experimental 
semester. In all classes, the course material used for the study spanned half the material covered 
during the semester. This encompassed 6 chapters from the required text, covered on the second 
and fourth of 4 tests during the semester. A total of 30 test questions were targeted for analysis, 
five from each chapter. Of the 30 questions from each chapter, 18 were factual questions, asking 
about definitions, steps in a process, or other facts about the material. The other 12 questions 
were conceptual, requiring application of the factual material to given situations. Although 
students were not alerted to any relationship between the PRS items and the test questions, the 
relationship was the independent variable used to create 3 experimental and 2 control conditions. 
The test questions were chosen carefully in order match, as closely as possible, their relative 
degree of difficulty within each treatment condition. All PRS questions used for the study were 
factual, asking only about basic information presented in class. 

The 5 test questions from each chapter were each used in one of the study’s 5 conditions. 
Sample items are provided in the Appendix. The Identical condition presented a factual test item 
in class as a PRS question. The Reworded condition contained factual PRS and test questions on 
the same topic, but the items were not identical to one another. Both the questions and the 
response choices differed. The Conceptual condition included a factual PRS question in class 
that probed the information relevant to a targeted conceptual test question. Conceptual test 
questions required students to apply a principle or fact to a hypothetical situation. The Control- 
Factual and Control-Conceptual conditions, respectively, presented factual or conceptual 
questions on the tests but no in-class PRS questions relevant to their content. Six of each item 
type (one from each chapter) were included in the study. 

The classes all met three times per week (Monday, Wednesday and Friday) for 50 
minutes at 11:00am. In fall 2007, the PRS items were spread across 7 weeks of a 15-week 
semester. The PRS questions appeared on slides as part of the PowerPoint presentations 
delivered in class each day. An average of 3-6 PRS questions were given in class each week with 
the items relevant to the study dispersed throughout the weeks in which the targeted chapters 
were discussed. As the instructor explained concepts or research findings, students were 
encouraged to ask questions or engage in discussion about the material. PRS questions were 
typically asked after a concept was presented and discussed, and only after students were 
encouraged to ask for clarifications or additional information. Some were asked as discussion 
starters and others for credit. A title at the top of a PRS slide indicated to students whether a 
given question was for credit or discussion. All PRS questions used in the present study were 
credited. 

When the instructor activates the iClicker system with the remote (or keyboard) a timer 
appears on the screen, allowing a time limit to be set for responses. Typically, 60-90 seconds was 


2 

Due to a test production error, one of the items in the Identical condition was left off the second exam. As such, the analyses for 
that condition are based on results from 5 items rather than 6. 
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allowed for factual questions. A bar graph showing the distribution of responses and the correct 
answer was displayed for another few seconds after the voting was “closed”. Students were 
encouraged to ask questions after seeing the graph and correct answer. If there was not high 
agreement (90% or greater) on the correct answer, another minute or so was spent discussing the 
item, whether students posed questions or not. On all but a few of the PRS items used in the 
study, however, students scored 90% or higher and asked no questions after seeing the correct 
answer. 

The tests in this class were not cumulative, each covering only the assigned material 
since the previous test. Both tests analyzed for the present study included 50 multiple-choice 
questions. Each test included 9 experimental test items (3 items from each of the experimental 
conditions) and 6 control questions (3 in each of the control conditions). A total of 4 tests were 
given during the semester, but only tests 2 and 4 were targeted for analysis. Test 2 was given 6 
weeks into the semester and test 4 was given on the last class day of the semester. 

C. Analyses. 

Attendance. Attendance data in the experimental semester were gathered from the 
iClicker data files, which maintains a record of the number of students submitting PRS responses 
per day. Those data were used to calculate the mean number of students in class each day over 
the semester. This figure was compared to prior fall semesters in which other incentives to attend 
class were offered. Attendance for those semesters was determined by calculating the mean 
number of papers handed in on days that papers were collected. In fall 2005 and 2006 the papers 
were for pop extra credit or pop quizzes, respectively, given roughly once per week in each 
semester. Both these classes were conducted in an identical fashion to the test semester, except 
that no PRS was used in either of those semesters. Only minor updates or changes were made in 
PowerPoint slides and lecture content, none of which would be expected to affect attendance. 

Learning. The study was conducted in a live classroom so it was not possible to employ a 
fully controlled experimental design. Because questions couldn’t be counterbalanced between 
PRS-paired and control conditions, item differences could be responsible for differences between 
conditions. Indeed, despite efforts to use items in each condition that were comparable in 
difficulty, there were differences between conditions in the baseline (No-PRS) semester in fall 
2006. Because of these differences, directly comparing scores between conditions in the PRS 
semester would not be particularly informative. The analysis was conducted in such as way, 
however, so as to reduce error stemming from item differences. Specifically, since all the test 
questions used in this study appeared on tests given to the No-PRS class, the percent of the class 
getting each question correct in the No-PRS semester could be used as a baseline measure. This 
allowed condition comparisons on the basis of percent improvement over the baseline semester 
rather than raw scores. Thus, the analyses were conducted on the improvement scores to mitigate 
item differences as a potential source of error. 

III. Results. 

A. Effect of PRS on Attendance. 

Attendance in the PRS class was equivalent to attendance in the pop quiz semester, 167 
(80%) and 165 (79%), respectively. Average attendance in the extra credit semester was 128 
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students (61%). The attendance rate in the extra credit semester was significantly different from 
the other semesters, = 9.36 g < .01. The classes employing pop quizzes and PRS each had 
attendance rates roughly 30% higher than the class that offered pop extra credit as an attendance 
motivator. In real terms, when pop quizzes or PRS were used instead of pop extra credit, roughly 
38 more students (18% of the class) on average came to class each day. 

B. Effect of PRS on Learning. 

Test items paired with PRS questions were correctly answered by 75% of the 
experimental class while only 62% of students in the No-PRS semester correctly answered the 
same questions. This is an increase of 20.9%. In contrast, an average of only 69% of students in 
the PRS semester correctly answered the control questions (those not paired with PRS 
questions), as compared to 67% of students in the No-PRS class. This is an improvement of only 
2.9%. The difference in performance increase between PRS-paired and control items is 
statistically significant, )^{\) = 13.5, p < 0.001 

Additional analyses were conducted to examine the effect of the PRS questions 
separately on factual and conceptual test items. Figure 1 illustrates the data for the factual test 
questions. It shows the mean percent correct for each condition in the PRS and No-PRS 
semesters with the percent difference between classes printed over each set of bars. The PRS 
class improved significantly more on PRS-targeted factual test questions than on control-factual 
items _p(^(2) = 43.9, p < 0.001. 
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Figure 1. Relative performance on the factual test items in the PRS and No-PRS classes. 

The percent performance increase between classes in each condition is printed above each set of 
bars. 



Figure 2 illustrates the results for the conceptual test questions. It shows the mean 
performance score for each class in the PRS-Paired and control conditions, with the percent 
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increase printed above the bars. The PRS class improved significantly more on conceptual PRS- 
targeted test questions than on control-conceptual questions, )^{\) = 1 1.7 p < 0.001. 

90.0 

80.0 
70.0 


60.0 

50.0 

% Correct 

40.0 

30.0 

20.0 
10.0 

0.0 


Figure 2. Relative performance on the conceptual test items in the PRS and No-PRS 
classes. The percent performance increase between classes in each condition is printed above 
each set of bars. 

IV. Discussion. 

PRS and paper-based pop quizzes both resulted in attendance rates of roughly 80%, 
which is 30% higher than attendance rates when paper-based extra credit opportunities were 
offered. In a class of just over 200 students, this translated to an average of 38 more students 
coming to class each day. Given that PRS did not enhance attendance more than paper-based pop 
quizzes, one may wonder if PRS is worth the effort. After all, there is no reason to believe that 
students indicating responses on paper would diminish the learning effects and PRS does require 
an initial time investment to learn the technology and create the question slides. However, the 
paper method requires the distribution, collection and grading of hundreds of papers each week. 
In contrast, PRS is simple to use and importing grades to grade books or files requires only a 
button click. Because of the large number of students the technology results in a net time saving, 
both in and outside of class. 

The PRS effects, however, were more profound than attendance alone. Students’ test 
performance demonstrated greater retention and comprehension of information targeted by PRS 
questions. In spite of the fact that the PRS questions were all factual, enhanced performance was 
observed for both factual and conceptual test questions. The effect can’t be attributed to the 
attendance increase because (1) performance on control items increased significantly less than on 
target items and (2) attendance was comparable between the PRS class and the No-PRS class 
used as a baseline. 
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It is clear that the benefit of PRS does not transfer to information that is not explicitly 
addressed by the PRS questions. That is, the learning effects were observed only for test items 
that were matched with PRS items and not to the control items. This point is important for two 
reasons. First, several prior studies used overall test score as a measure of PRS effectiveness. In 
discussing those reports, I proposed that PRS effects may have been diluted because items not 
targeted by PRS were included in the dependent measure. The results reported here support that 
contention. Based on the isolated effects of PRS, I do suggest that future studies isolate PRS- 
targeted items as a dependent measure in order to get a clearer picture of the technology’s effect. 
The second implication of this finding relates to pedagogy. Specifically, the result suggests that a 
sufficient number of PRS questions should be offered to have a meaningful effect on overall 
learning. Because the effects of PRS do not transfer to untargeted information the technology’s 
value is limited unless it is used across topics. Fortunately, the technology allows a question to 
be asked, answered and scored in class within 60-90 seconds. Thus, the efficiency of the 
technology allows a number of questions to be asked each day, each targeting key lecture points 
as they are presented. 

Another advantage of PRS over in-class, paper-based assessment is the instant feedback 
provided to students. Because the correct answer and class performance are projected instantly to 
the class, the instructor is able to reinforce comprehension or correct misconceptions 
immediately. Epstein et al. (2002) cite the immediate feedback function of PRS as one of the 
technology’s major advantages. Indeed, not only are students able to learn by having their 
misconceptions addressed, but students who are inattentive in class and incorrectly answering 
questions may be motivated by seeing that their performance puts them in a small minority of the 
class. Again, it is certainly possible to offer questions in class each day and offer feedback once 
the papers are collected. The manual method, however, does not make the class performance on 
each item available to the instructor or students in the moment, when the material is fresh. 

The magnitude of the effect reported here is greater than that in some other studies and 
this is likely due to two methodological factors. First, the present study carefully matched the 
PRS items to assessment items in the present study. As noted earlier, other studies have 
compared overall test performance between PRS and no-PRS classes (e.g., Morling et al., 2008; 
Ribbens, 2007) without isolating test questions that probed information addressed by the PRS 
questions. Since there is no reason to suspect that PRS use would affect retention of information 
unrelated to the material actually addressed by PRS questions, including those items may have 
diluted the effect. Indeed, the control items included in the present study demonstrate that PRS 
effects do not transfer to untargeted information. Another difference between the study and some 
others is that the present study took place in a live classroom rather than a laboratory and PRS 
performance was factored into final grades. As such, motivational differences exist between the 
present study and those conducted in a laboratory or offering PRS as extra credit. Students 
should be more strongly motivated to score well on PRS items and test questions when they 
directly affect their course grades. 

While the present methodology has some advantage over laboratory studies, it is a 
concern that the experimenter taught the classes used for the present analyses. Experimenter bias 
is always a concern in research, so it is important that future investigations replicate the present 
study, thereby validating the results as well as the methodology. The quasi-experimental design 
also limits the strength of the conclusions. However, one would expect that uncontrolled 
variables between classes that could have affected test performance would largely affect control 
and PRS-paired items equally. Since the amount of change from the No-PRS semester was 
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markedly smaller for the control items than the PRS-paired items, the most likely source of the 
difference is the PRS questions. 

Another potential criticism of the present study is that the PRS items may be effective 
simply because they provided test questions in advance. Indeed, had the present study shown 
only that students did better on test questions in the Identical condition than the control 
conditions, the results would be relatively mundane. After all, one would expect students to do 
better on test questions they had previewed in class. What is convincing about the results is that 
the Reworded and Conceptual test items also showed significant performance gains. In other 
words, asking students about a topic in class allowed them to better retain the information and 
score higher on items that had never before been encountered. That the effect held for novel 
factual and conceptual test questions is all the more compelling. 

There are two competing theoretical explanations for the effect of PRS on learning. The 
first possibility is that the PRS questions merely highlight important ideas for students. In other 
words, asking questions about particular facts signals to students which topics the instructor 
views as important. As such, the effect may come about by prompting students to direct 
attentional resources to specific items during class and in subsequent study. The second 
possibility is that retrieval is acting as a source of memory encoding. Known as the testing effect, 
it has been shown that the act of recalling a piece of information can strengthen it in memory 
(Carrier and Pashler, 1992; Roediger and Karpicke, 2006). As such, it is possible that, by asking 
students to retrieve a piece of information in the moments just after encoding it, PRS questions 
help students solidify memory for the relevant information. A study currently underway attempts 
to find support for one of these explanations over the other. If the testing effect is the source of 
the PRS effect on test performance, it would mean that PRS technology offers a true learning 
advantage rather than mere study prompts. Such a result would be important to our 
understanding of both learning theory and pedagogical practice. 

In spite of the positive results of the data reported here there are some reasonable 
concerns about implementing PRS and it is important to acknowledge them. Among those 
concerns is the initial time investment to learn and set up the system. If one is currently using no 
technology in the classroom, I suggest starting slowly by moving to electronic presentations such 
as PowerPoint before attempting a PRS system. There is an initial time investment to learn the 
software and understand how the receiver and software integrate with the computer, PowerPoint 
and grade books. Many campuses have already standardized on a system or at least have a 
number of instructors using a particular system. In this case, I do recommend taking advantage 
of campus computing services or inquiring with one’s colleagues before getting started. Once 
comfortable with electronic presentations and the PRS software, adding PRS requires very little 
time or effort, as it requires only the addition of a few slides for each class. Questions can even 
be taken from test question banks provided by the text publisher. Once those items are created, 
they may be refined and others may be added each semester. Since the scoring and recording of 
grades is automatic, the initial time commitment to learn the software and create PRS slides 
should be recouped after the first semester of use. 

Some may be concerned about technical difficulties and I have read reports describing 
technical challenges of working with PRS, including difficulty registering students, insufficient 
bandwidth, faulty remotes and other problems (Hatch et ah, 2005). There are several excellent 
systems on the market today and there is no reason to suffer with a poor system. In my 
experience, the technology works with very little setup time or difficulty and is available for both 
Apple and PC platforms. The receiver supplied by the company has a range of 200 feet so the 


Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 1, January 2009. 


22 



Shapiro, A. 


size of the room should not be an issue in most cases. I do suggest conferring with computing 
support services and colleagues on one’s campus for advice prior to adopting a system. If one’s 
campus has standardized on a particular system it will be supported on campus and local help 
will be available to individual instructors. If one’s campus does not offer technical help, I 
suggest choosing a system that comes with the promise of reliable technical support online or 
over the phone. Again, one’s colleagues are a good source for such information. 

Students do indeed have a cost associated with the system. They are required to purchase 
a remote just as they are required to purchase textbooks or pay lab fees in some classes. 
However, students are able to sell back their clickers to the campus bookstore, just as they do 
with their books. As PRS becomes more common, many universities are standardizing on single 
systems. Standardization is an excellent idea because it allows students to use their “clickers” 
across classes and years. 

Students are also given added responsibilities when PRS is implemented in a course. 
They must register their clickers to get credit for their responses. With the system I adopted, 
registration takes just a minute or two per student. Students register simply by entering their 
name, student number and PRS serial number on a website. They are also required to come to 
class each day with their remotes and are given the responsibility to remember them. The PRS 
brand used in the present study, iClicker, does provide faculty with an extra remote and the 
ability to loan one to a student in class. With such a large class, however, I recommend a “zero 
tolerance” policy wherein a student gets no response credits on a given day if he or she comes 
unprepared. Otherwise, it will become a daily hassle to temporarily register the loaner remote 
and choose between multiple students asking to borrow it. 

Faculty may be concerned about academic honesty with PRS. It is not possible for 
students to cheat by lending remotes to one another because a remote can only be registered to 
one person. However, just as there is no way to prevent all cheating on tests, there is some 
opportunity to cheat with PRS. A student skipping class could give his or her remote to a 
classmate to enter responses for him or her. I have made it clear that any student found with two 
remotes will be disciplined for cheating, as will the owner of the second remote. Nonetheless, I 
am quite sure I have not prevented this practice completely. 

In spite of the cost and responsibilities passed on to students I am aware of no study that 
reports students disliking PRS in their courses. The present study did not assess student 
perception of the technology, but students’ informal feedback was very positive. Moreover, other 
studies that explicitly asked students about their attitude toward the technology have reported 
positive responses from students (Hatch et ah, 2005; Stowell and Nelson, 2007; Trees and 
Jackson, 2007). 

In sum, the research presented here improves on prior methodology by studying PRS in a 
live classroom and by pairing assessment items with PRS questions. The data indicate that 
students benefit from PRS because they are motivated to attend class and learning outcomes are 
significantly improved. There is an initial time commitment to learn the software and create the 
PRS items. After that commitment is met, however, PRS gives instructors the ability to engage 
students, keep them motivated and focused, and enhance learning for little additional time and 
effort. 


The instructor records the registration by creating a text file of student names and clicking a button on a screen in the software 
to “sync” the registration records with the roster. After that, all the responses are available to the instructor in a variety or 
formats. 
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Appendix 1. Examples of Stimuli and Test Items in the 3 Experimental and 2 Control 
Conditions. Test items in the control conditions were similar to those in the experimental factual 
and conceptual conditions but were not paired with any PRS questions on their respective topics. 
The study included 6 items for each condition, one from each chapter covered on the tests. 

Identical 

PRS Item Test Item 


The critical factor in the context effect is the 
influence of 

A. maintenance rehearsal. 

B. retroactive and proactive interference. 

C. external environmental cues in a 
particular situation. 

D. cryptomnesia. 


The critical factor in the context effect is the 
influence of 

A. maintenance rehearsal. 

B. retroactive and proactive interference. 

C. external environmental cues in a 
particular situation. 

D. cryptomnesia. 

Reworded 


PRS Item Test Item 


Tolman found that rats that were first 
rewarded on the 1 1^'’ day for finishing a maze 
did just as well on the 12* day as those who 
were rewarded every day. This result is 
important because: 

A. It tells us that reinforcement is always 
vitally important to learning 

B. It tells us that learning can happen 
without reinforcement 

C. Partial reinforcement is necessary for 
learning 

D. You can always tell how much a rat has 
learned by watching its behavior. 


Psychologist Edward C. Tolman's studies with 

rats in mazes led him to conclude that: 

A. reinforcement is not necessary for 
learning to occur. 

B. learning will not occur in the absence of 
reinforcement. 

C. rats learn nothing more than a sequence of 
left and right turns. 

D. continuous reinforcement is necessary for 
operant conditioning to occur. 


Conceptual 


PRS Item 

A primary reinforcer is 

A. something we do not have to be taught to 
like. 

B. something that we find intrinsically 
rewarding. 

C. often related to food, safety or comfort. 

D. All of the above. 


Test Item 

“I'll make you a deal,” Leroy's mother says. “If 
you clean up your room, then you can have a 
glazed donut.” Using operant conditioning 

terms, Leroy's mother is using to reward 

desired behavior. 

A. punishment by avoidance 

B. a conditioned reinforcer 

C. a primary reinforcer 

D. negative reinforcement. 
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None 


None 


Control-Factual 

PRS Item Test Item 

Punishment is most effective if: 

A. it immediately precedes the operant. 

B. it consistently follows the operant. 

C. it occasionally follows the operant. 

D. there is considerable delay between the 
operant and the punishment. 

Control-Conceptual 

PRS Item Test Item 

Jonathan frequently plays the slot machines 
and sometimes comes out slightly ahead in his 
winnings. Like all gambling behavior, 

Jonathan's gambling behavior is on a 

schedule of reinforcement. 

A. fixed-ratio 

B. fixed-interval 

C. variable-interval 

D. variable-ratio 
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